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ABSTRACT 

A  program  is  described  that  implements  the  "centroid"  algorithm  for  the 
optimization  of  functions  of  many  variables  by  a  stochastic  direct-search 
procedure.  The  program  is  available  as  a  Windows  application  with  context- 
sensitive  on-line  help,  and  can  maximize  any  suitable  function  exported  by  a 
dynamic-link  library  (DLL).  For  practical  reasons  of  simplicity,  the  program  is 
limited  to  real  functions  of  up  to  20  real  variables,  and  the  search  domain  is 
limited  to  hyper-rectangular  blocks  in  multi-dimensional  Cartesian  spaces.  The 
only  constraint  on  the  function  to  be  optimized  is  that  it  should  be  single-valued 
at  every  point  in  the  domain,  and  should  be  capable  of  being  evaluated  in  a  DLL. 
It  need  not  be  smooth  or  continuous.  A  sample  program  is  provided  to  illustrate 
the  generation  of  a  suitable  DLL  using  Borland  Pascal.  In  general,  the  algorithm 
employed  will,  given  a  sufficient  number  of  search  iterations,  find  the  global 
maximxim  of  a  function  in  the  presence  of  closely  competing  local  maxima. 
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A  Windows  Program  for  Multivariate 
Optimization 


EXECUTIVE  SUMMARY 


A  recent  paper  by  the  author  and  a  colleague  (Benke  and  Skinner,  1991) 
describes  a  new  method,  the  "centroid"  algorithm,  for  finding  the  maximum 
value  within  a  given  domain  of  a  function  of  many  variables.  This  method  is 
based  on  a  theory  of  noise  reduction,  and  is  applicable  to  any  type  of  function 
that  is  single-valued  and  finite  at  every  point  in  the  domain,  the  only  restriction 
being  that  it  should  be  capable  of  being  calculated  in  a  computer  program. 
Being  a  statistical  process,  the  method  does  not  require  the  function  to  be  either 
analytically  or  numerically  differentiable,  or  to  have  only  a  single  peak  in  its 
domain,  but  unlike  most  statistical  algorithms  does  not  need  any  arbitrary 
scaling  or  "cooling"  parameters.  Finally,  the  method  is  relatively  insensitive  to 
the  number  of  variables,  unlike  Monte  Carlo  techniques  where  computing  effort 
varies  exponentially  with  this  factor. 

The  referenced  paper  describes  a  number  of  test  applications  of  the  centroid 
algorithm,  including  a  function  subject  to  random  fluctuations  much  greater 
than  the  underlying  variance,  and  a  13-dimensional  function  where  advantage 
was  taken  of  symmetries  in  the  function  definition.  Another  problem 
successfully  solv^  elsewhere  involved  a  simple  function  of  100  000  variables. 
However,  it  was  felt  that  there  was  a  need  for  a  general-purpose  implementation 
of  the  algorithm  for  convenient  solution  of  the  most  common  case,  a  function  of 
a  few  (up  to  20)  variables  in  a  continuous  Cartesian  space,  with  no  advantage 
taken  of  symmetries.  That  is  what  this  program  implements. 

The  approach  taken  was  to  require  the  user  to  program  a  dynamic-link  library 
(DLL)  with  the  function  to  be  optimised.  Given  a  suitable  language  compiler, 
which  includes  at  least  some  versions  of  Pascal,  C++,  Fortran  and  Common  Lisp, 
this  is  a  relatively  simple  operation;  in  Borland  Pascal  version  7.0  it  requires  only 
five  lines  of  code  additional  to  the  definitions  of  the  function  and  the  type  of 
variable  domain.  This  approach  was  preferred  to  the  provision  of  build-in 
functions  and  interpretation  of  equations  entered  at  run-time,  since  it  gives 
much  greater  flexibility,  and  users  of  multi-dimensional  optimisation  are  very 
likely  to  be  proficient  in  at  least  one  of  the  suitable  languages.  The  report 
contains  an  example  of  a  suitable  DLL,  which  optimises  a  function  used  in  Benke 
and  Skinner  (1991). 


The  program  is  implemented  in  Windows,  with  an  interface  of  a  type  that  will 
be  familiar  to  most  users.  A  dialog  box  is  used  to  select,  by  bringing  up  a 
standard  file-open  dialog,  a  DLL  for  the  function  to  be  optimised,  and  to  enter 
the  name  of  the  function  and  the  number  of  variables.  This  causes  a  table  for 
minimum  and  maximum  values  to  expand  to  accommodate  all  the  variables. 
When  the  table  has  been  filled  in,  together  with  optional  valties  for  the  number 
of  iterations  of  the  algorithm  and  for  a  random-number  seed,  the  optimisation 
can  be  run  any  number  of  times.  While  it  is  running,  there  is  a  continuously 
changing  display  of  the  niunber  of  iterations  completed  and  the  current  best 
estimates  for  the  optimised  function  and  its  position  in  the  variable  domain.  It  is 
possible  to  switch  between  function  DLLs,  or  between  functions  in  a  DLL, 
during  a  single  session  with  the  program.  The  program  uses  a  standard  form  of 
Windows  context-sensitive  on-line  help,  with  the  usual  hypertext  links  and  pop¬ 
up  definitions. 
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1.  Introduction 


This  report  describes  a  Windows  application  program  for  the  optimization  of  a 
multimodal  function  in  many  dimensions,  and  provides  a  sample,  in  Borland  Pascal, 
to  illustrate  how  a  dynamic-link  library  (DLL)  may  be  written  to  define  the  function. 
The  "centroid"  algorithm  used  is  described  by  Benke  and  Skinner  (1991),  and  briefly 
summarized  in  Appendix  A.  Essentially,  the  algorithm  involves  repeated  generation 
of  random  sets  of  parameters,  within  the  constraints  of  the  problem,  and  produces 
weighted  averages  (centroids)  in  a  way  designed,  according  to  a  simple  theory  of  noise 
reduction,  to  produce  a  set  that  minimizes  the  deviation  from  the  optimum.  As 
described  in  the  reference,  the  algorithm  has  been  applied  successfully  to  a  number  of 
problems  varying  from  simple  two-dimensional  functions  to  complicated  functions  of 
several  hundred  variables,  ^though  this  implementation  is  limited  to  20  variables.  For 
reasons  of  simplicity  in  defining  the  function,  no  use  is  made  of  the  potential  reduction 
in  computing  effort  that  could  be  achieved  by  taking  advantage  of  s)anmetries  of  the 
function  in  the  search  domain. 

An  obvious  question  is,  how  can  the  program  possibly  know  how  to  do  anything, 
when  it  has  no  information  on  the  problem?  The  answer  is  that  it  provides  the  basic 
algorithm  and  all  the  necessary  housekeeping,  and  it  is  up  to  the  user  to  provide  one 
or  more  DLLs,  each  defining  one  or  more  functions  to  be  maximised.  For  many 
problems,  writing  the  function  is  not  a  particularly  onerous  chore,  and  an  example  is 
given  in  the  source  code  provided  in  Appendix  B 


2.  Document  Conventions 


Where  an  instruction  refers  to  keys  joined  by  +  signs,  these  are  to  be  entered 
simultaneously.  Where  references  to  keys  are  joined  by  commas,  they  are  to  be 
entered  consecutively.  Thus 

enter  Alt+F,  S 

means  press  the  F  key  whUe  the  Alt  key  is  held  down,  release  both  keys  and  then 
press  the  S  key.  The  use  of  capital  letters  here  does  not  imply  pressing  the  Shift  key. 
When  an  instruction  refers  to  menu  selections  separated  by  commas,  each  is  contained 
in  a  sub-menu  of  the  previous.  Thus 

select  File,  Set  Up  from  the  menu  system 

means  use  the  moxise  to  click  the  word  File  on  the  main  (permanently  displayed) 
menu,  then  click  the  words  Set  Up  on  the  sub-menu  that  will  appear  in  response  to  the 
first  click. 
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Fixed-pitch  type,  such  as  centroid.exe,  is  used  where  it  is  necessary  to  refer  to 
program  file  names,  or  to  identifiers  within  program  files. 


3.  Description  of  the  Program 


3.1.  Hardware  and  Software  Requirements 


The  program  requires  to  be  run  imder  Windows  3.0  (or  later),  and  the  user  must  have 
a  language  compiler  capable  of  generating  DLLs  (this  includes  at  least  some 
implementations  of  Pascal,  C++,  Fortran  and  Common  Lisp).  The  program  will  run 
much  faster  if  the  computer  is  equipped  with  an  arithmetic  co-processor. 


3.2  Program  Files  and  Sub-Directories 


The  program  is  provided  on  a  distribution  disk,  copies  of  which  are  available  on 
request  from  the  Chief,  Maritime  Operations  Division,  at  the  address  on  the  title  page. 
It  takes  the  form  of  six  files,  which  are: 

(a)  centroid.exe.  This  is  essentially  just  a  user  interface,  written  in  Visual  Basic. 
It  is  used  to  input  library  and  function  names,  and  other  data  relevant  to  an 
optimisation.  It  also  maintains  a  watching  brief  on  the  optimisation,  and  presents 
continually  revised  data  on  progress. 

(b)  cent.dll.  This  provides  the  algorithm,  together  with  its  housekeeping  and 
interaction  with  the  Windows  Application  Program  Interface.  It  was  written  in 
Borland  Pascal. 

(c)  centroid. hip.  This  provides  standard  Windows  on-line  help,  with  the  usual 
pop-up  definitions,  cross-referenced  jumps  and  search  facilities.  The  help  is 
context-sensitive,  that  is,  at  any  time  the  user  can  select  a  Help  button  or  menu 
item,  or  press  the  FI  key,  and  be  presented  with  the  help  topic  most  likely  to  be 
relevant  to  the  current  state  of  the  program.  It  was  written  as  a  rich-text  format 
(rtf)  file,  using  Microsoft  Word,  and  converted  to  a  help  file  using  the  Microsoft 
help-file  compiler  (hep .  exe). 

(d)  cmdlalog.vbx,  commdlg.dll,  threed.vbx.  These  are  standard  Visual 
Basic  DLLs  that  are  used  for  generating  various  dialog  boxes  and  for  accessing  the 
help  file.  (The  copyright  to  these  files  is  held  by  the  Microsoft  Corporation,  but 
distribution  with  applications  written  in  Visual  Basic  is  permissible.) 
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To  instal  the  program  automatically,  insert  the  distribution  disk,  select  File,  Rtm 
from  the  Windows  Program  Manager  menu  system,  and  type 

a:\setup 

in  the  space  labelled  Command  Line.  If  the  disk  is  not  in  drive  A,  change  the  first 
character  of  the  command  accordingly.  The  set-up  program  will  ask  for  a  sub¬ 
directory  name  for  the  installation,  and  will  suggest  c :  \centroid  as  a  default.  It 
will  copy  the  two  main  files,  centroid.exe  and  centroid. hip,  into  this,  and  the 
remaining  files  into  a  sub-directory  determined  by  yoiu-  Windows  set-up  (if  they  are 
not  already  there).  Finally,  it  will  generate  a  group  containing  icons  for  the  program 
and  its  help  file. 

If  Windows  is  running  from  a  network  and  the  sub-directory  for  the  DLLs  is  write- 
protected,  the  automatic  set-up  program  cannot  be  used.  In  this  case,  or  if  you  want  to 
control  over-writing  of  existing  DLL  files,  the  files  may  be  copied  manually  from  the 
a :  \manual  sub-directory  to  any  sub-directories  of  your  choice,  bearing  in  mind  that 
the  help  system  expects  to  find  centroid. hip  in  the  same  sub-directory  as 
centroid.exe,  and  that  the  remaining  files  should  be  in  the  default  library-search 
path.  You  wiU  then  need  to  generate  one  or  two  program  items,  and  optionally  a 
program  group,  using  the  techniques  described  in  the  Windows  manual.  During 
manual  set-up,  you  can,  if  you  wish,  change  the  main  part  of  the  file  names  of  the  exe 
and  hip  files  provided  these  remain  the  same  as  each  other,  but  the  remaining  four 
files  must  retain  their  original  names. 

If  the  Browse  button  (see  later)  is  used  for  finding  DLLs,  a  small  file  named 
centroid.ini  is  written  into  the  Windows  directory.  If  you  have  changed  file 
names  during  manual  set-up,  this  file  will  have  a  corresponding  change.  The  file 
enables  the  browse  algorithm  to  start  looking  in  the  sub-directory  where  the  last  DLL 
was  found,  even  if  this  was  during  a  previous  session  with  the  program. 


4.  Writing  the  Merit  Function 


In  order  to  find  the  maximum  value  of  a  function,  it  is  necessary  to  provide  a 
dynamic-link  library  defining  the  function.  The  DLL  is  dynamically  linked  to  the  main 
program  at  run  time,  which  means  that  the  user  can  switch  between  libraries,  or 
functions  within  the  libraries,  to  perform  a  number  of  optimisations  in  one  session. 
The  name  of  the  library  within  the  file  is  irrelevant. 

The  merit  function  may  be  programmed  in  any  language  that  supports  DLLs.  It  must 
return  a  real  value  express^  in  the  IEEE  four-byte  floating-point  standard,  which  has 
various  names  in  these  languages,  as  shown  in  Table  1.  The  only  parameter  to  the 
function  should  be  a  contiguous  group  of  the  same  type  of  variable,  called  by 
reference.  This  may  be  an  array,  or  any  other  type  of  record  or  structure  as  defined  for 
the  language  in  use  -  the  example  in  Appendix  B  has  a  Pascal  user-defined  record 
type.  Multi-dimensional  arrays  are  not  recommended,  since  the  order  in  which  their 
elements  are  assumed  to  be  stored  in  memory  depends  on  the  language  used. 
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Table  1.  Names  of  Real  Types  in  Various  Languages 


Language 

Name  of  Four-Byte  Real 

Pascal 

single 

Fortran 

real* 4 

C++ 

float 

Common  Lisp 

float 

The  program  can  recognise  a  function  either  by  its  name,  which  is  not  case  sensitive, 
or  by  its  ordinal  number.  The  techniques  for  assigning  exported  names  and  ordinal 
numbers  to  functions  in  DLLs  are  described  in  the  manuals  for  the  various  languages. 
The  example  in  Appendix  B  shows  how  an  ordinal  number  is  assigned  in  Borland 
Pascal  -  it  is  the  number  following  the  word  index  in  the  exports  statement.  In  this 
case,  no  exported  name  is  specified,  so  it  will  default  to  the  identifier 
(MultiGaussian)  used  in  the  code. 

In  Borland  Pascal,  it  is  necessary  to  use  the  { $N+ ,  E+ }  compiler  directives  to  access 
the  single  type  and  to  allow  the  program  to  be  run  on  a  machine  without  a 
mathematics  co-processor.  It  is  also  desirable,  in  the  interests  of  generating  readable 
code,  to  define  a  record  type  (rather  than  an  array  type)  for  the  parameter,  and  to 
use  a  with  clause  in  the  merit  function.  Appendix  B,  which  implements  the  multi- 
Gaussian  function  of  the  reference,  demonstrates  the  form  of  a  suitable  Pascal  DLL.  It 
can  be  seen  from  this  example  that  (apart  from  non-functional  comments  and  blank 
lines)  the  source  code  contains  only  the  definition  of  the  parameter  set,  the  function  to 
be  optimised  and  five  additional  lines.  These  are  the  library  statement,  a  compiler- 
directive  comment,  an  escorts  statement  and  the  two  lines  of  an  empty  initialisation 
block. 


5.  Running  the  Program 


When  the  program  is  started,  the  user  is  faced  with  a  blank  desk-top  and  a  menu 
containing  two  items.  File  and  Help.  The  Help  item  gives  access  to  a  standard 
Windows  help  file  that  is  an  abbreviated  version  of  this  report.  The  File  item  contains 
three  options,  labelled  Set  Up,  Run  and  Exit,  with  the  Run  option  initially  disabled. 


5.1  The  Set  Up  Option 


The  program  starts  off  with  no  information  about  what  function  is  to  be  optimised, 
about  the  number  of  values  in  the  parameter  set  passed  to  the  DLL,  or  about  the 
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domains  that  these  values  are  drawn  from.  In  order  to  run  an  optimisation,  therefore, 
it  is  necessary  to  set  up  a  number  of  conditions.  To  do  this,  you  can 

select  File,  Set  Up  from  the  menu  system, 
enter  Alt+F,  S, 
enter  Ctrl+S,  or 
press  the  Return  key. 

This  will  bring  up  the  parameter  set-up  dialog  which  is  initially  as  shown  in  Fig.  1. 
The  top-left  section  (dark  blue  text  on  most  screens)  contains  edit  boxes  for  the  library 
and  function  names,  the  number  of  parameters,  a  random-number  seed  and  the 
maximum  number  of  iterations  for  the  algorithm.  The  top-right  section  (brown  or 
dark  red  text  on  most  screens),  has  edit  boxes  for  the  minimum  and  maximum  values 
for  all  parameters.  Initially,  there  is  room  for  only  one  parameter.  The  third  section 
contains  buttons  labelled  Accept,  Cancel  and  Help. 

To  set  the  function  for  optimisation,  first  enter  the  library  name  in  the  top  edit  box, 
which  initially  displays  * .  dll.  It  is  not  necessary  to  include  the  extension  if  this  is 
.dll.  If  no  path  is  entered,  the  program  carries  out  a  default  library  search  (for  more 
details,  see  the  on-line  help).  Alternatively,  you  can  select  the  Browse  button  (or  enter 
Alt+B)  to  search  for  the  DLL  file  using  a  standard  file/load  dialog.  This  has  the 
advantage  that  you  will  not  need  to  type  in  the  file  name,  or  you  may  indeed  have 
forgotten  the  name,  but  you  will  still  need  to  remember  either  the  function  name  or  its 
ordinal  number,  and  will  need  to  know  the  number  of  parameters  and  the  domains 
from  which  they  are  drawn.  Once  the  browse  facility  has  been  used,  the  dialog  box 
will  always  start  in  the  sub-directory  from  which  the  last  DLL  was  read,  even  if  the 
program  has  since  been  closed.  In  the  second  edit  box,  you  can  enter  either  the  ordinal 
number  or  the  name  of  the  function  -  the  default  is  ordinal  number  1. 

To  set  the  number  of  parameters,  simply  edit  the  third  box,  and  the  number  of  lines 
in  the  second  section  will  change  to  accommodate  the  value  entered.  If  your  entry 
cannot  be  interpreted  as  an  integer  in  the  range  1  to  20,  an  error  message  will  appear. 

There  is  no  need  to  change  the  random-number  seed,  which  is  initially  zero. 
However,  changing  this  results  in  a  different  set  of  pseudo-random  parameter  values, 
to  allow  for  different  starting  points  on  the  random  walk  through  parameter  space. 
Any  value  entered  here  must  be  capable  of  interpretation  as  a  long  integer,  i.e.  a 
standard  four-byte  signed  integer. 

The  maximum  number  of  iterations  is  initially  set  at  1000.  This  is  usually  adequate 
for  a  moderately  complicated  function  of  a  few  variables,  but  it  can  be  edited  to  any 
number  from  1  to  the  maximum  for  a  long  integer.  Note  that  this  is  a  maximum  only  - 
iteration  can  be  stopped  at  any  time  without  loss  of  information. 

After  the  number  of  parameters  has  been  selected,  minima  and  maxima  for  these  can 
be  entered  in  the  appropriate  edit  boxes,  in  the  same  order  that  they  are  defined  in  the 
DLL.  These  entries  must  be  capable  of  interpretation  as  single  values.  The  random 
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parameter  values  will  be  selected  uniformly  from  these  ranges,  so  the  parameter 
domain  is  effectively  a  hyper-rectangular  solid  in  a  multi-dimensional  Cartesian  space. 

A  completely  filled-in  set-up  dialog,  for  the  example  in  Appendbc  B,  is  shown  in 
Fig.  2.  TTie  use  of  the  function  name  in  the  second  edit  box  is  redundant  and  piuely 
for  demonstration  -  the  default  ordinal  value  1  could  have  been  left  untouched. 

After  the  dialog  box  is  completely  filled  in,  the  user  may  select  either  the  Accept  or 
Cancel  button.  Tlie  Accept  button  may  be  selected  from  the  keyboard  using  Alt+A  or 
Enter,  and  the  Cancel  by  Alt+C  or  Escape.  As  is  conventional,  the  Accept  button 
changes  the  program  values  to  those  in  the  dialog  box,  and  the  Cancel  buttons  clears 
the  dialog  box  without  making  any  changes. 


5.2.  The  Run  Option 


To  run  an  optimisation, 

select  File,  Run  from  the  menu  system, 
enter  Alt+F,  R,  or 
enter  Ctrl+R. 

This  cannot  be  selected  until  you  have  accepted  the  set-up  dialog  box  at  least  once. 
When  it  is  selected,  it  brings  up  the  progress-report  window,  which,  when  there  are 
two  parameters  as  in  Appendix  B,  is  initially  as  shown  in  Fig.  3. 

Start  the  optimisation  by  selecting  the  Start  button  (or  Alt+S  or  Enter).  The  window 
labelled  Iteration  will  display  the  number  of  times  the  program  has  cycled  through  the 
algorithm,  until  either  you  select  the  Stop  button  (or  Alt+P  or  Escape),  or  the  iteration 
count  reaches  the  maximum  set  in  the  set-up  process.  The  window  labelled  Parameter 
Values  will  display  the  current  best  estimates  for  all  the  parameter  values,  changing  as 
the  estimates  improve. 

The  window  labelled  Summary  of  Last  Ten  Changes  shows  the  iteration  count  each 
time  the  current  best  estimate  for  the  parameter  set  improves,  the  function  value  at 
each  change,  and  the  algorithm  status  for  that  change.  This  latter  value  can  be  Initial 
(black  text  on  most  screens)  if  the  parameters  are  the  initial  random  set.  Learned  (dark 
green  text)  if  the  set  resulted  from  successful  application  of  the  algorithm,  or  Random 
(red  text)  if  the  last  random  parameter  set  produced  the  current  best  estimate. 
Usually,  after  an  initial  settling-in  period,  the  status  will  be  overwhelmingly  Learned. 
Figure  4  shows  the  progress-report  window  shortly  after  the  start  of  an  optimisation. 

After  the  optimisation  is  complete,  you  can  either  re-select  Start  or  select  Exit  (or 
Alt+X).  Re-selecting  Start  causes  a  new  run  starting  from  scratch,  but  with  the 
random-number  generator  running  on  from  its  last  call.  This  will  give  a  new  random 
walk  through  parameter  space,  but  with  the  original  values  for  parameter  limits  and 
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maximum  number  of  iterations.  Selecting  Exit  allows  you  to  run  the  set  up  again,  or 
to  exit  the  program. 


5.3  Leaving  the  program 

If  either  the  set-up  or  progress-report  window  is  displayed,  select  Cancel  or  Exit  as 
appropriate.  When  you  can  see  only  the  main  window, 

select  File,  Exit  from  the  menu  system, 
enter  Alt+F,  X, 
enter  Alt+F4,  or 
press  the  Escape  key. 

You  can  also  leave  by  using  the  system  menu  box  to  close  the  main  window. 


6.  What  Types  of  Function  can  be  Optimized? 


The  function  to  be  optimized  needs  to  be  capable  of  being  evaluated  in  a  computer 
program,  and  to  be  single  valued  at  all  points  in  the  search  domain,  otherwise  there 
are  no  restrictions.  Within  these  constraints,  the  function  need  not  be  analytic,  smooth 
or  continuous  -  one  problem  solved  by  Benke  and  Skinner  (1991)  used  a  function 
subject  to  random  noise. 

Because  the  optimization  is  a  stochastic  process,  it  is  not  possible  to  analyze  its 
behaviour  for  different  merit  functions  (although  some  theory  is  given  in  the 
reference).  It  is  possible,  however,  to  demonstrate  that  the  process  will  behave  like  an 
inefficient  Monte-Carlo  search  if  the  maximum  value  of  the  merit  function  is  close  to 
zero  in  a  domain  that  spans  zero  -  the  analysis  is  restricted  to  functions  that  are 
everywhere  positive  and  finite.  If  there  is  a  problem  with  slow  convergence,  it  is 
suggested  that  the  user  try  to  re-state  the  merit  function  so  that  (a)  it  is  everywhere 
positive,  and  (b)  there  is  a  large  variation  of  merit  function  over  the  search  domain, 
since,  contrary  to  what  one  might  intuitively  expect,  convergence  can  be  fastest  when 
the  global  peak  occupies  a  relatively  small  part  of  the  domain.  There  is  some 
suggestion,  from  observation  of  various  types  of  function,  that  convergence  is  best  for 
isotropic  parameter  sets,  that  is,  sets  in  which  aU  parameters  perform  similar  functions 
and  all  are  of  comparable  size. 
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7.  Summary 


This  report  describes  the  practical  implementation,  as  a  Windows  application  with 
context-sensitive  on-line  help,  of  the  centroid  algorithm  defined  by  Benke  and  Skinner 
(1991).  It  is  presented  in  a  form  such  that  the  user  need  only  provide  a  function 
definition  in  a  dynamic-link  library.  An  example  is  provided  to  illustrate  how  this  is 
achieved  in  Borland  Pascal. 


Reference 


K.K.  Benke  and  D.R.  Skinner,  A  direct  search  algorithm  for  global  optimization  of 
multivariate  functions,  Australian  Computer  J.,  Vol.  23,  no  2,  pp  73-85,  May,  1991. 
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Figure  1.  Parameter  set-up  dialog  box  in  initial  state. 


Figure  2.  Parameter  set-up  dialog  box  after  data  entry. 
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Appendix  A 

Description  of  the  Centroid  Algorithm 


This  Appendix  represents  a  very  brief  summary  of  the  reference.  The  problem  is, 
given  a  merit  function,  f{X),  of  a  set  of  parameters,  represented  as  a  vector  X,  to  find 
the  value  Xopt,  within  a  given  domain,  for  which  the  function  is  maximtun.  We 
consider  only  functions  that  are  everywhere  positive  and  finite. 

We  can  consider  any  vector  X  as  the  sum  of  Xopt  and  an  error  vector  A,  which  in  turn, 
in  the  absence  of  any  information,  is  considered  to  be  pxu-ely  random  in  nature.  We 
now  try  to  estimate  a  weighting  factor  a,  in  the  range  0.0  to  1.0,  such  that  we  can  form 
a  weighted  average  of  two  values  of  X  so  as  to  minimize  the  resultant  A.  That  is,  given: 

Xi  =  X^+Ai 

and 

X2  =  Xopj  +  Aj  ^^2) 

find  the  value  of  a  such  that  I  A3 1  is  minimum,  when 


X3  =  aXi  +  (l-a)X2  =  X^,+A3 


(A3) 


It  is  shown  in  the  reference  that,  provided  the  values  of  A  are  truly  random. 


a=IA2lV(IAiP+IA2p) 


(A4) 


that  is,  the  two  vectors  X  should  be  weighted  inversely  as  the  squares  of  their  error 
vectors.  Clearly,  this  is  not  possible  in  practice,  since  we  would  need  to  know  the 
value  of  Xopt  to  calculate  a.  However,  if  we  assume  that  there  is  a  negative  correlation 
between  the  function  value  /(X)  and  I A I ,  we  can  at  least  make  sure  that  the  weighting 
is  in  the  right  direction  by  weighting  the  vectors  directly  as  the  function  values.  This 
is  the  basis  of  the  centroid  algorithm.  We  can  continually  generate  random  samples  of 
X,  compute  /(X),  and  take  a  weighted  mean  with  the  current  best  X,  i.e.  that  with  the 
highest  value  of  /(X)  so  far  foimd.  If  either  the  random  or  the  averaged  set  gives  a 
higher  merit  function,  this  becomes  the  new  best  X.  The  first  value  of  X  used  would 
normally  be  random.  It  is  shown  in  the  reference  that,  if  ideal  weighting  could  be 
achieved,  the  rate  of  decrease  of  A3  would  be  expected  to  be  independent  of  the 
munber  of  dimensions  in  X. 


The  algorithm  based  on  the  above  considerations  is  shown  schematically  in 
Figure  Al.  Some  comment  is  required  on  the  criterion  for  stopping  the  search.  Since 
the  search  is  a  stochastic  process,  it  is  difficult  to  define  a  stopping  criterion  in  terms 
of,  for  example,  the  derivative  of  the  best  solution  with  respect  to  iteration  number. 
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We  have  therefore  provided  two  criteria,  which  are  (a)  the  exceeding  of  an  iteration 
count,  and  (b)  manual  intervention  by  mouse  or  from  the  keyboard. 


Figure  Al.  Schematic  diagram  of  the  centroid  algorithm. 
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Appendix  B 


Sample  Somce  Code  for  Dynamic-link  Library 


(****  Function  for  Testing  Visual  Basic  Version  of  Centroid  Algorithm  **** 


This  Is  the  multl-Gausslan  function  used  In  Benke  a  Skinner,  Australian 
Coa^uter  J.  vol.  23  no.  2  pp.  73-85,  May  1991. 

Language  Is  Borland  Pascal  ver  7.0.  To  compile  with  Turbo  Pascal  for  Windows 
ver  1.5  omit  the  $E+  compiler  directive. 


****  Author  David  Skinner,  AMRL  ver  1.1  November  1994  ***) 
library  example;  {  This  name  Is  not  used  by  programs  accessing  the  DLL  } 
[$N-i-,E+  For  access  to  Single  type  and  (If  necessary)  emulation  } 


type 

vector  -  record  {  Definition  of  parameter  set  } 

X  :  Single; 
y  :  Single 

end; 


function  MultlGausslan (var  param  :  vector)  :  Single;  export; 

{  Definition  of  function  to  be  maximized  } 

var 


j  ;  Word; 


log_term, 

sum 

const 


Single; 


{  Counts  number  of  Gaussians  } 
{  Logarithm  of  current  term  } 
{  Sum  of  Gaussians  } 

(  Constants  used  for  generation  of  multl-Gausslan  } 


a 

:  array 

[1..5] 

of  Single  - 

( 

0.5, 

1.2, 

1.0, 

1.0, 

1.2 

>; 

xi 

:  array 

[1..5] 

of  Single  - 

( 

0.0, 

1.0, 

0.0, 

-0.5, 

0.0 

); 

yi 

:  array 

[1..51 

of  Single  - 

( 

0.0, 

0.0, 

-0.5, 

0.0, 

1.0 

); 

si 

:  array 

[1..5] 

of  Single  - 

{ 

0.1, 

0.5, 

0.5, 

0.5, 

0.5 

} ; 

log 

-  20; 

{  Maximum 

absolute  value  of  parameter 

to  Exp  ) 

begin 

sum  0.0; 

for  j  1  to  5  do  with  param  do 
begin 

log_term  (Sqr(x  -  xi[j])  +  Sqr(y  -  yi[j]))  /  Sqr(si[j]); 

if  log_term  <-  max_log  then  sum  sum  +  a[j]  *  Exp(-log_terra) 


end; 

MultlGausslan  sum 


end; 


exports  MultlGausslan  index  1; 


{  Making  the  function  accessible  } 


begin 

end. 
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